Where Does the Time Go in Software DSM Systems: Experiences with JIAJIA?

The performance gap between software DSM systems and message passing platforms prevents the prevalence of software DSM system greatly, though great eeorts have been delivered in this area in the past decade. In this paper, we take the challenge to nd where should we focus our strength on in the future design. The components of total system overhead of software DSM systems are analyzed in detail rstly. Based on a state-of-the-art software DSM system JIAJIA, we measure these components on Dawning parallel system and draw ve important conclusions which are diierent from some traditional viewpoints. (1) The performance of the JIAJIA software DSM system is acceptable. For four of eight applications, the parallel eeciency achieved by JIAJIA is about 80%, while for two others, 70% eeciency can be obtained. (2) 40.94% interrupt service time is overlapped with waiting time. (3) Encoding and decoding diis do not cost much time(<1%), so using hardware support to encode/decode diis and send/receive messages is not worthwhile. (4) Great endeavours should be put to reduce data miss penalty and optimize synchronization operations, which occupy 11.75% and 13.65% of total execution time respectively. (5) Communication hardware overhead occupies 66.76% of the whole communication time in the experimental environment, and communication software overhead does not take much time as expected. Moreover, by studying the eeect of CPU speed to system overhead, we nd that the common speedup formula for distributed memory systems does not work under software DSM systems. Therefore, we design a new speedup formula special to software DSM systems , and point out that when the CPU speed increases the speedup can be increased too even if the network speed is xed, which is impossible in message passing systems. Finally, we argue that JIAJIA system has desired scalability.